Computational Sample Complexity and Attribute-eecient Learning Author to Whom Proofs Should Be Sent

Two fundamental measures of the e ciency of a learning algorithm are its running time and the number of examples it requires (its sample complexity). In this paper we demonstrate that even for simple concept classes, an inherent tradeo can exist between running time and sample complexity. We present a concept class of 1-decision lists and prove that while a computationally unbounded learner can learn the class from O(1) examples, under a standard cryptographic assumption any polynomial-time learner requires almost (n) examples. Using a di erent construction, we present a concept class of k-decision lists which exhibits a similar but stronger gap in sample complexity. These results strengthen the results of Decatur, Goldreich and Ron [9] on distribution-free computational sample complexity and come within a logarithmic factor of the largest possible gap for concept classes of k-decision lists. Finally, we construct a concept class of decision lists which can be learned attribute-e ciently and can be learned in polynomial time but cannot be learned attribute-e ciently in polynomial time. This is the rst result which shows that attribute-e cient learning can be computationally hard. The main tools used are one-way permutations, error-correcting codes and pseudorandom generators. 3 List of symbols: capital theta capital omega Q product (capital pi) epsilon delta alpha tau x x bar D calligraphic D I calligraphic I T calligraphic T S calligraphic S C calligraphic C G calligraphic G U calligraphic U Z calligraphic Z O capital oh 0 zero o lowercase oh ` lowercase ell 1 one 4 1 in nity h left angle i right angle circle ! right arrow asterix subset 2 element of z }| { overbrace 5